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ABSTRACT 

The purposes of th.s study were to test for both 
forward and backward effects of inserted questions in prose material, 
and to determine the conditions under which ef=ich in most effective. 
The conditions examined were the relationship between the inserted 
questions and the subsequent questions on which learning facilitation 
was revealed, and the textual distance of the material tested from 
the inserted questions. Subjects for the experiment were 140 
undergraduate students who were randomly assigned to one of five 
treatment groups: inserted questions (EQ) at three varying positions 
in the reading material (El, E2, E3) , no inserted questions (C) , and 
inserted questions irrelevant to the preceding material (CQ) . The 
treatments used formed two seperate studies. Groups El, C, and CQ 
comprised one study (A) and groups E2 and E3 the other (B) . The 
results from study A indicated no significant difference between C 
and CQ. Thus, the effect of increased attentiveness occurs only with 
the insertion of text related questions. In study B the results 
indicated that group E3 was significantly superior on matched items, 
regardless of distance from the inserted question, but there was no 
significant difference on unmatched items. (WR) 
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SHAPING AND/OR REVIEW FUNCTIONS OF QUESTIONS IN PROSE MATERIAL 



The insertion of questions in prose material has been shown to 
facilitate learning from the material (Rothkopf, 1966). The effects 
of these questions are both direct, facilitating subsequent perform- 
ance on Identical questions, and indirect, facilitating performance on 
other questions about the text materials. The effect is marked when 
the questions refer to preceding material in the text (post-questions) 
but may even be reversed when they refer to subsequent material (pre- 
questions) (Frase, 1968; Frase, Patrick and Schumer, 1970; Rothkopf, 
1966). 

Although both direct and indirect effects have been reported, the 
operation of these effects has not been clearly explicated. One con- 
jecture -is that the questions serve to shape inspection behaviors thus 
facilitating performance on posttest items dealing with material fol- 
lowing the questions in the text (Rothkopf, 1963). That iS| the effec 
of the questions is forward in that they influence the inspection of 
materials that have not yet been read. From this point of view inspec 
tion behaviors are seen to be reinforced (and, thus, maintained) if th 
inserted questions can be answered, or non-reinforced (and, thus, 
altered) if the inserted questions cannot be answered. A number of 
studies have strengthened this shaping hypothesis. Rothkopf and 
Bisbicos (1967) observed that the facilitation was greater towards the 
end of the text and that it was selective, being greatest for items in 
the criterion test which were similar to those in the original text. 
Rothkopf and Coke (1968) found that learning was an increasing func- 
tion of the likelihood that fragments of the text were noticed. Frase 
(1969), in demonstrating the effects of different organizations of 
prose material, showed that learning was detenrdned by the aspects of 
the text to which the learner could attend. 

Such a forward effect need need not operate only through shaping 
appropriate inspection behaviors, causing Ss to attend to appropriate 
features of the text. It may also function by simply controlling the 
level of attentiveness, causing the reader to attend more carefully to 
the material following each set of inserted questions. The effect of 
the questions would be cyclic, with the effect diminishing as the 
reader moves through the material following the questions but being 
reinstated following the next set of questions. An explanation only 
in terms of shaping appropriate inspection behaviors would predict a 
cumulative improvement rather than a cyclic effect. Although there 
are no clear data to support this second hypothesis there is some 
suggestive evidence. Rothkopf and Bloom (1970), for example, found 
that reading rate slowed after each set of inserted questions. 

The two hypotheses discussed above postulate a forward facilita- 
tive effect for post-questions. However, the experimental results to 



date have not ruled out the possibility of the indirect facilitative 
effect of questions being a backward, or review, effect* Frase (1968) 
noted that the superiority of post-questions over pre-questions occur- 
red even on the first paragraph (though this could not be attributed 
to a suppressive effect of pre-questions). Watts and Anderson (1971) 
found no increase in performance towards the end of their material. 
Bruning (1968) demonstrated that there is an additive review compo- 
nent in the effect of post-questions* 

If the facilitative effect of inserted questions is, at least in 
part, a review effect, then the facilitation could be expected to be 
greater for criterion test items which deal with material related to 
that reviewed in answering the inserted questions* Rothkopf and 
Bisbicos' (1967) observation of greater facilitation on criterion test 
items similar to questions inserted in the text could be accounted for 
in this manner. Further, such a review effect could be expected to 
be stronger with short preceding lags, that is, with material read 
shortly before the inserted questions and weaker with longer lags* 

If the facilitative effect of inserted questions is, at least in 
part, a respondent phenomenon — with the effect of the questions 
increasing the attentiveness of material following the questions — then 
the effect should be greatest with short following lags, that is, on 
material immediately following the questions and weaker with longer 
lags. 

The purpose of the present study was to test for both forward 
and backward effects of inserted questions, and to determine the con- 
ditions under which each is most effective. The conditions examined 
were the relationship between the inserted questions and the subse- 
quent questions on which facilitation was revealed, and the textual 
distance of the material tested from the inserted questions. 

METHOD 

Materials 

The basic material used in the reported experiments was a selec- 
tion of material from Rachel Carson's book. The Sea Around Us. The 
text was multilithed onto 21 pages of approximately 260 words each. 
From each page of the material three questions were developed* All 
63 questions were of the completion type and required the recall of 
specific information from the text. The questions were prepared so 
that, for each page, tv^o of the questions dealt with the same mate- 
rial, while the third dealt with an unrelated topic* 

The pairs of questions dealing with the same' material were 



developed in such a way that, although they dealt with the same text 
material, neither could be answered from a knowledge of the answer to 
the other. For example, from the following passage in the text a pair 
of questions were formed. 

' Then from the surveying ship Bulldog^ examining a 
proposed northern route for a cable from Faroe to 
Labrador in 1860^ aame another report. The 
Bulldog ^s sounding line^ wh'^^rjh at one place had 
been allowed to lie for same time on the bottom 
at a depth of 'J 260 fathoms ^ aame up with 13 star- 
fish clinging to it. 

The two questions developed we-.e* 

(a) The surveying ship which recovered starfish 
from a depth of 1260 fathoms in 1860^ was 
eccploring a route for a cable from Faroe to 



(b) Th^ surveying ship ^ which recovered 

starfish from a depth of 1260 fathoms in 1860^ 
was exploring a route for a cable from Faroe. 

The third question developed from the same page as this pair required 
the name of an Arctic explorer. 

Thus 42 of the 63 questions were matched in pairs. Of these 42, 
21 (one chosen at random from each pair) were selected for insertion 
in the text material for the various experimental treatments. These 
inserted questions are referred to as EQs. The remaining 42 items, 
21 matched and 21 unmatched with the EQs were used to form a criterion 
test (Tl). The 21 EQs were also used in a criterion test {T2) given 
after Tl. 

Test Tl was intended to measure the general facilitative effect 
of EQs whereas test T2 was to measure specific learning of the materi- 
al tested by EQs. The two tests were bound into a single test booklet. 

A general measure of reading ability was obtained for all 5s by 
administration of Part II (Reading) of the Reading Comprehension Test 
(Form lA) from the E.T.S. Cooperative English Tests. 
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Treatments 

Three experimental treatments and two control treatments were 
used. All text materials were bound into booklets which were distri- 
buted in random order, and all Ss took part in the experiments at the 
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sanie time. Because Ss could see one another they were told explicitly, 
ill the printed instructions at the front of the booklet, that, 
although all booklets contained the same passages, come were arranged 
differently from others and that they should not be concerned if other 
i^s appeared to be involved in writing, for example, when they were not. 
The instructions to all Ss were to '*study each page of the chapter 
carefully, paying close attention to facts and figures and to names 
and dates.** 

The treatments, summarized in Figure 1, were as follows: 

El. Experimental Group 1; After the first six pages, i.e., 
after Sections A and B at position ab, a sheet was bound into the 
text booklet with the six EQs from those pages. Similar sheets, each 
with three EQs from the preceding three pages, were bound into the 
text booklet after the ninth, twelfth and fifteenth pages, i.e., at 
positions c, d, and n, respectively. Thus, inserted questions always 
referred to preceding material. 

E2. Experimental Group 2; Questions were inserted in the 
text at positions ab, d, and f in a manner similar to that for El. 

E3. Experimental Group 3: Questions were inserted in the 
text at positions c, e, and g. 

C. Control Group ; No insertions were included in the 
text booklets. After the instructions, Ss simply read the 21 pages 
of text. 

CQ. Control Group with Questions ; At the same points in 
the text as for the El group, a corresponding number of irrelevant 
questions {CQs) from a Personal Opinion Scale, adapted from a dogma- 
tism scale (Rokeach, 1960), were introduced. These questions 
required approximately the same time to complete as the EQ items. The 
purpose of this treatment was to determine the effect of providing 
break points in the reading without text related questions. 

Subjects 

Subjects for these experiments consisted of 140 undergraduates, 
obtained as paid volunteers, at Concordia Teachers College, River 
Forest, Illinois. Vwenty-eight Ss were assigned at random to each of 
the treatment groups. 

Procedure 

All Ss attended a single group session. The reading comprehen-* 
sion test, requiring 25 minutes, was administered first. The experi- 
mental booklets were then distributed to Ss^ who worked through them 
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at their own pace. When a S completed hi.s booklet, he indicated this 
to a monitor who removed it and provided him wit'i a test booklet con- 
taining both tests Tl and T2u Ss were allowed to leave the room when 
they had completed both tests. The entire procedure required about 
90 minutes for the slower 5s. 



RESULTS AND DISCUSSION 

The treatments used in this study form two separate experiments. 
Groups El, C and CQ comprise one experiment (Experiment A) '^nd groups 
E2 and E3 the other (Experiment B) . The results for these a*. a pre- 
sented separately. 

EXPERIMENT 

Facilitative Effect of I nserted Questions 

Mean scores, raw and adjusted, on the two criterion tests Tl and 
TS, are shown in Table 1. The adjustment resulted from the use of 

TABLE 1 

Mean Scores on Criterion Tests by Groups 
for Experiment A 

Tl T2 

Group Raw '^'^an Adjusted Mean Raw Mean Adjusted Mean 

El 15.7 16.0 8.14 8.27 

C 13.5 13.5 4.25 4.23 

CQ 14.1 13.8 4.64 4.52 



reading comprehension test scores as a covariate. (Test T2 actually 
contained 21 items but, for this analysis, only those 15 inserted in 
the text for group El were used.) For test Tl^ the differences among 
the adjusted means were significant, F(2,80) = 3.53, p < .04. 

The difference between the two control groups was not significant. 
Thus, there was no support for the hypothesis that the facilitative 
effect of inserted questions was due to increased attentiveness fol- 
lowing a rest from reading. That is, the effect apparently occurs 
only with the insertion of text related questions such as the EQs. 
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On test T2 the performance of group El was similarly superior to 
that of the control groups, F(2,80) = 25.86, p < .001. Again the dif- 
ference between the control groups was not significant. Control group 
CQ was, therefore, dropped for subsequent analyses. 

Forward or Backward Effect 

Groups El and C were compared on the 12 items in Tl from sections 
A and B (the first six pages, read before any questions were encoun- 
tered) and the 12 items from sections F and G (read after the last 
questions had been encountered). A forward effect would be expected 
to produce superiority of El on the FG itoms while a backward effect 
should rpoduce superiority of El on the AB items. The adjusted means 
are given in Table 2. 

TABLE 2 

Adjusted Means on AB and FG Items in Test Tl 
by Groups for Experiment A 



Groups 







Items 






AB 


FG 


Overall 


El 


2.05 


4.86 


3.46 


C 


1.80 


4.31 


3.05 


Overall 


1.93 


4.59 


3.25 



The difference between the groups was not significant, F(l,53) « 
1.72, p > .18. Although the overall difference between scores on AB 
and FG was significant, F(l,54) = 98.15, p < .001, it is of little 
substantive importance since no attempt was made to control for dif- 
ferences in item difficulty. The substantively important groups by 
sections (AB, FG) interaction was not significant, F(l,54) - .11. 

There are theoretical grounds, however, for believing that for- 
ward and backward effects might operate in a more limited fashion than 
could be revealed in such a gross analysis as that shown above. A 
review effect should be greater for criterion icems matched with the 
inserted questions and, in particular, for matched items testing mate- 
rial from pages which ittunediately preceded the inserted questions. An 
attentional increase following inserted questions would, on the other 
hand, exert the greatest effect on material from pages immediately 
following the inserted questions — an effect which should be revealed 
by both matched and unmatched items. 
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In order to test these hypotheses performance on all criterion 
test (Tl) questions dealing with material from pages immediately before 
and immediately after inserted questions was considered. Pages 6, 9, 
12 and 15 were those "before" insertions and pages 7, 10, 13 and 16 
were those "after" insert ionr.. From each of these pages there were 
two questions on test TI9 one matched and one unmatched with one of 
the questions on the adjacent insert. Mean performances for the 
groups are shown in Table 3. 

TABLE 3 

Adjusted Means on Matched and Unmatched Items 
Before and After Inserts by Groups for Experiment A 

Matched Unmatched 

Groups Before After Before After 

El 1.59 2.05 1.95 2.41 

C 1.01 1.87 1.79 1.87 

Overall 1.30 1.96 1.87 2.14 

Analysis of covariance (Table 4) revealed group El to have been 
significantly superior to group C in overall performance, F(l,53) = 
4.02, p < .05. The significant effect for position does not 

TABLE 4 

Analysis of Covariance for Matched and Unmatched Items 
Before and After Position of Inserts for Experiment A 

Source SS df MS F P 

Betioeen subjects 



Groups [G] 


7.44 


1 


7.44 


4.02 


.047 


Subjects within 












groups [S(G)] 


98.05 


53 


1.85 






■thin suhjeots 












Position of items [P] 


12.07 


1 


12.07 


14.67 


.000 


PxG 


.00 


1 


.00 


.00 




PxS(G) 


44.42 


54 


.82 






Type of item [I] 


7.87 


1 


7.87 


13.89 


.001 


IxG 


.01 


1 


.01 


.03 


.854 


IxS(G). 


30.60 


54 


.56 






IxP 


2.16 


1 


2.16 


3.74 


.055 


IxPxG 


2.16 


1 


2.16 


3.74 


.055 


IxPxS (G) 


31.17 


54 


.57 
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demonstrate that performance on items from pages after questions was 
superior. Since the P x G interaction effect was not significant, the 
position effect was due only to differences in item difficulty. The 
same is true for the significant effects for item type and position x 
item type interaction. 

The important effect in this analysis is the I x P x G inter- 
action effect. Although the test of this effect fell just above the 
conventional level of significance, F(l,54) = 3.74, p < .055, it is 
examined here in some detail since a supplementary analysis of the 
data in Experiment B, reported later, showed the effect to have been 
replicated there. This result indicates that the groups by position 
of items interaction for matched items was significantly different 
from that for unmatched items. ThR effect is shown in Figure 2, from 
which the trend in the data can cleavly be seen. Tests of simple main 
effects (Winer, 1962, p. 323) showed that El was significantly superior 
to C on matched items from pages immediately before inserted questions, 
F(l,215) = 4.96, p < .05, and on unmatched items from pages immediately 
after inserted questions, F(l,215) = 4.30, p < .05. 

For the unmatched items there is little likelihood of a review 
effect operating. The results observed with unmatched items can 
readily be accounted for in terms of a forward effect. After each set 
of inserted questions Ss appeared to attend more carefully to the text, 
hence the superiority of El over C on itetns from the first page after 
each insert. On continuing to read the ^t tenti^^eness to the text pre- 
sumably diminished with a consequent dro- in the relative level of 
performance of El. Such a pattern can be seen in the data when it is 
recognized that pages referred to as "before" inserts were three pages 
after the prior insert* fur the unmatched items it seems more useful 
to distinguish the page positions as "shortly after" and "long after" 
rather than "after" and "before." El was superior to C on questions 
from pages shortly after inserts but the effect was attenuated with 
increasing textual distance from the insert — the difference between 
the groups on the "long after" pages being insignificant. 

Such a forv/ard effect should also operate with the matched items 
but, with these items, there is the additional possibility of a review 
effect facilitating performance on material from pages prior to the 
inserts. The data for the matched items suggest that such a review 
effect did^ in fact, operate. El was significantly superior to C on 
matched items from pages immediately prior to inserts but not on pages 
after the inserts. These "after" pages, for consideration of a review 
effect, are better refarred to as "long before" inserts. Just as the 
facilitative forward effect on the unmatched items was attenuated with 
increased textual distance after the insert, the facilitative review 
effect on matched items was attenuated with increased textual distance 
after the insert, the facilitative review effect on matched items was 
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EXPERIMENT B 

The experiment with groups E2 and E3 was designed to provide a 
further test of the alternative forward and backward hypotheses. The 
design of this experiment can be seen in Figure 1. For group E2 
inserted questions occurred before Sections C, E, and G whereas, for 
group E3 they occurred after these sections. 

Forward or Backward Effect 

The adjusted mean performances of the two groups, on the items 
in test Tl which were drawn from the pages in C, E, and G, are shown 
in Table 5, The data from which these means were obtained were 
analyzed in a repeated measures ANOVA with covariance adjustments on 
the between subjects variable. This analysis r.howed that the overall 
difference between the groups was not significar t, F(l,53) = .013. 
The groups by sections interaction effect, however, was significant, 
F(2,108) = 6,81, p < ,002, 

TABLE 5 

Adjusted Means on Sections C, E, and G by Groups for Experiment B 
Group Section C Section E Section G 

E2 2.27 3.22 2*33 

E3 1.75 3.84 2.16 



Tests of the simple main effects (Winer, 1962, p. 311) for groups 
for each section of the material, revealed that the superiority of E2 
on Section C approached significance, 2^(1,161) = 3,1, p < .10, that 
the superiority of E3 on Section E was significant, F(l,161) « 4.4, 
P < .05, and that there was no significant difference between the 
groups on Section G. 

These data provide important information about the nature of the 
forward and backward effects. On Section C, prior to which only 
group E2 had received inserted questions, the performance of E2 was 
superior. The inspection behaviors of Ss in E2 had apparently been 
shaped, on seeing the earlier questions, so that they attended to the 
specific factual information tested by the inserted questions and the 
criterion test. The extent to which inserted questions can have a 
facilitative shaping effect depends on the extent to which Ss habit- 
ual inspection behaviors are inappropriate for the particular text 
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material. Whether the effect would be less marked with questions of a 
different type from those used in the present study largely remains to 
be shown. Watts and Anderson (1971), in fact, reported data sugges- 
tive of a review effect rather than a shaping effect with "application 
questions . 

Prior to Section E, on which the performance of E3 was superior, 
both groups had encountered inserted questions. This superiority was 
obtained despite the fact that group E2 had encountered two sets of 
inserted questions, including one immediately prior to the section. 
The superiority of E3 can, therefore, be attributed to a review effect 
occurring because of the inserted questions immediately after the 
section. Thus, it appears that inserted questions may serve a review 
function only if appropriate Inspection behaviors have been used. 

The non-significant difference on the final Section G cannot be 
accounted for but an explanation might be due to a recency effect 
obliterating the advantage to E3 of having questions at the end of 
the text immediately prior to taking the criterion test. 

Replication of Results of Experiment A 

The primary analysis for Experiment B, reported above, indicated 
that both shaping and review effects occur. The data in Experiment A 
suggested that the review effect was greatest with matched items. A 
further analysis was made of the data from Experiment B to determine 
whether the effect on matched and unmatched questions had been repli- 
cated. In order to do this the data from control group C In Experi- 
ment A was included. 



For group E2 the pages immediately before inserts were 6, 12 and 
18 and those after were 7, 13, and 19. These constitute page group 
A in Table 6 in which the adjusted mean performances of the groups 

TABLE 6 

Adjusted Means on Matched and Unmatched Items 
Before and After Inserts by Groups for Experiment B 

Before After 

Matched Unmatched Matched Unmatched 

Page Group A E2 1.40 1.31 1.38 1.63 

(6,7,12,13, 

18,19) C 1.09 1.33 1.43 1.18 

Page Group B E3 1.28 1.20 .72 .90 

(9,10,15, 

16,21) C .92 1.43 .44 .95 
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are shown. Page group B is the set of pages before (9, 15, 21) and 
after (10, 16) for group E3, 

The analysis for groups E2 and C, with data from page group A, 
produced precisely the results noted in Experiment A, viz a signifi- 
cant groups X page position x jtem type interaction, iP(l,54) = 6.07, 
p < .02. The graph of this interaction, in Figure 3, shows the effect 
to be the same as before, superior performance for the group with 
inserted questions on matched items from pages immediately prior to 
inserted questions, F(l,215) = 2.1, p < .15, and on unmatched items 
immediately after inserted questions, F(l,215) = 4.4, p < .05. The 
analysis for groups E3 and C revealed a non-significant groups x page 
position X item type interaction, F(l,54) = .07, but, in this case, 
the groups x item type interaction was significant, ■?(1,54) = 3,94, 
P < .05. Group E3 was significantly superior on matched items regard- 
less of distance from the insert but there was no significant differ- 
ence on unmatched items. 



CONCLUSIONS 

Previous research on the effect of inserted questions has shown 
both direct and indirect facilitation of subsequent performance on 
questions relating to the text. .The indirect facilitation has occur- 
red on items to which, it had been demonstrated, there was no direct 
transfer model in which knowledge of the answer to one question facil- 
itates answering another. In this study the matched questions were 
constructed in such a way that the answer to one could nnf in any way 
provide the answer to the other member of the pair. Yet review of the 
material required to answer an inserted question facilitated perfor- 
mance on its matched item, provided that the review occurred within a 
page or so of the relevant material. 

The nature of the retrieval phenomenon, or memory search, which 
gives rise to the facilitation is not yet clear. The important 
variable could be similarity of subject matter though, in this study, 
the similarity occurred only in the subject matter of the sentence 
from which the responses were deleted, not in the responses themselves. 
Alternatively the important variable could be proximity of material in 
the text, with greater f acilit.-'tion for contiguous material. A further 
possibility is that verbatim rer.all of the original sentence to com- 
plete the inserted question provided also the word required to complete 
the matched question, despitf. the fact it was also deleted from the 
inserted question. Not all (^f the questions required ^simple verbatim 
recall but further research Vvith controlled use of verbatim and para- 
phrase items will clarify this issue. 

This study also confirmed the existence of a forward effect on 
the unmatched items. The results of Experiment B provide support for 
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a shaping hypothesis, suggesting that, insofar as 5s habitual inspec- 
tion behaviors are Inappropriate, inserted questions will serve to 
shape appropriate behaviors. The use of questions testing highly 
specific factual information probably highlighted this effect. The 
results of Experiment A suggested that, in addition to a shaping 
effect, the inserted questions serve also to control general atten- 
tional behaviors. Ss performed better on material from pages immedi- 
ately after the inserts. Such control of attention, however, was only 
achieved with text related questions. 
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